Self-Paced Learning: An Implicit Regularization Perspective
نویسندگان
چکیده
Self-paced learning (SPL) mimics the cognitive mechanism of humans and animals that gradually learns from easy to hard samples. One key issue in SPL is to obtain better weighting strategy that is determined by the minimizer functions. Existing methods usually pursue this by artificially designing the explicit form of regularizers. In this paper, we focus on the minimizer functions, and study a group of new regularizers, named self-paced implicit regularizers that are derived from convex conjugacy. Based on the multiplicative form of half-quadratic optimization, convex and non-convex functions induced minimizer functions for the implicit regularizers are developed. And a general framework (named SPL-IR) for SPL is developed accordingly. We further analyze the relation between SPLIR and half-quadratic optimization. We implement SPL-IR to matrix factorization and multi-view clustering. Experimental results on both synthetic and real-world databases corroborate our ideas and demonstrate the effectiveness of implicit regularizers.
منابع مشابه
A Self-Paced Regularization Framework for Multi-Label Learning
In this brief, we propose a novel multilabel learning framework, called multilabel self-paced learning, in an attempt to incorporate the SPL scheme into the regime of multilabel learning. Specifically, we first propose a new multilabel learning formulation by introducing a self-paced function as a regularizer, so as to simultaneously prioritize label learning tasks and instances in each iterati...
متن کاملSelf-Paced Learning with Diversity
Self-paced learning (SPL) is a recently proposed learning regime inspired by the learning process of humans and animals that gradually incorporates easy to more complex samples into training. Existing methods are limited in that they ignore an important aspect in learning: diversity. To incorporate this information, we propose an approach called self-paced learning with diversity (SPLD) which f...
متن کاملOn Convergence Property of Implicit Self-paced Objective
Self-paced learning (SPL) is a new methodology that simulates the learning principle of humans/animals to start learning easier aspects of a learning task, and then gradually take more complex examples into training. This new-coming learning regime has been empirically substantiated to be effective in various computer vision and pattern recognition tasks. Recently, it has been proved that the S...
متن کاملDevelopmental differences in effects of task pacing on implicit sequence learning
Although there is now substantial evidence that developmental change occurs in implicit learning abilities over the lifespan, disparate results exist regarding the specific developmental trajectory of implicit learning skills. One possible reason for discrepancies across implicit learning studies may be that younger children show an increased sensitivity to variations in implicit learning task ...
متن کاملSelf-Paced Multitask Learning with Shared Knowledge
This paper introduces self-paced task selection to multitask learning, where instances from more closely related tasks are selected in a progression of easier-to-harder tasks, to emulate an effective human education strategy, but applied to multitask machine learning. We develop the mathematical foundation for the approach based on iterative selection of the most appropriate task, learning the ...
متن کامل